Image Based Interactive Rendering with View Dependent Geometry

In this paper we present a novel approach for interactive rendering of virtual views from real image sequences. Combining the concepts of light fields, depth-compensated image warping and view dependent texture mapping, this plenoptic modeling approach can handle large and complex scenes. A portable, handheld multi-camera system has been developed that allows to record multiple image streams by simply walking around the scene. These image streams are automatically calibrated and depth maps for all views are generated as input to the rendering stage. For rendering a view dependent warping surface is constructed on the fly and depth-compensated image interpolation is applied with view-dependent texture mapping. Rendering quality is scalable to allow fast preview and to achieve high-end quality with the same approach. The system can handle large and geometrically complex scenes with hundreds of real images at interactive rates.